Roadmap of Generative Models

A roadmap of understanding modern generative models, including VAE, GAN, diffusion, and flow matching.

Almost all generative probabilistic model share a same core idea. There’s a true distribution \(p_{\theta}(\cdot)\) of a category of data in the world (e.g., a true distribution of all images, of all audios, etc.). If we know this true distribution, then we’re able to sample from this distribution to generate the data we want.

For any probabilistic model that trains to approximate a distribution, Maximum Likelihood Learning is always a good tool to use. It provides a standard way to train generative models.